Random processes via the combinatorial dimension: 

introductory notes 
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Abstract 

This is an informal discussion on one of the basic problems in the theory 
of empirical processes, addressed in our preprint "Combinatorics of random 
processes and sections of convex bodies", which is available at ArXiV and from 
our web pages. 

Arguably, the central problem of the theory of empirical processes is the follow- 
ing: 

Describe the classes of functions on which the classical limit theorems of 
probability hold uniformly. 

Precisely, for a given probability space one looks at the sequence of indepen- 

dent samples (Xj) with values in Q and distributed according to the law /x. Then 
the problem is to describe the classes F of functions / : Q — > M for which the se- 
quence of real valued random variables f{Xi) satisfies the classical limit theorems of 
probability uniformly over f £ F. The two classical limit theorems we have in mind 
are the law of large numbers, for which such classes are called Glivenko-Cantelli, 
and the central limit theorem, which gives rise to Donsker classes. 

In practice one does not know the law // according to which the samples Xi are 
drawn. Thus a particularly vital question is to know what classes are Glivenko- 
Cantelli or Donsker for every ^. Such classes are called universal or even uniform if 
the convergence in the corresponding limit theorems is uniform over all /i. We refer 
the reader to Chapter 14.3 of the book of Ledoux and Talagrand |LTj for a brief 
introduction to the theory of empirical processes and to the book of Dudley |Du 99j 
for a comprehensive account. 
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Vapnik and Chervonenkis were first to realize that the problem above is inti- 
mately connected with the combinatorics of the class F. In their pioneering works 
|VC 681 IVC 71[ IVC 81j , they quantified the size of a class F by what now is called 
the Vapnik- Chervonenkis dimension. This dimension makes sense only for Boolean 
classes, i.e. for classes of {0, l}-valued functions. A natural extension of this notion 
to general classes is possible (although not unique); the resulting concept is called 
by Talagrand "the quantity of fundamental importance" |T 02] . This quantity mea- 
sures how much F oscillates. 

Precisely, for a given t > 0, a subset a of 17 is called t-shattered by a class F if 
there exists a level function h on a such that, given any partition o" = (T_ U 0"+ , one 
can find a function f G F with f{x) < h{x) if x G (T_ and f{x) > h{x) if x G (T+. 
The combinatorial dimension of -F, denoted by v{F,t), is the maximal cardinality 
of a set t-shattered by F. Simply speaking, v{F,t) is the maximal size of a set on 
which F oscillates in all possible ibt/2 ways around some level h. 

For Boolean classes, this is clearly the classical Vapnik-Chervonenkis dimension 
in the whole nontrivial range < t < 1. For general classes, the combinatorial 
dimension appears implicitely in the work of Talagrand |T 92j and explicitely in the 
paper of Alon et al }ABCHj . 

Intuitively, the smaller the combinatorial dimension of a class F, the less F 
oscillates, hence the better chances are for F to be a Donsker class. Recalling the 
known methods such as Dudley's integral inequality and Sudakov minoration, it is 
not difficult to locate the obstacles for the class to be universal Donsker and, in fact, 
to guess the optimal description of such classes: 

Conjecture 1 For every uniformly bounded class F, 

/•oo 

/ \/v{F,t) dt < oo ^ F is universal Donsker ^ v{F,t) = 0{t ). 
Jo 

The history of the work related to this conjecture goes back to 70's. The right 
hand side, which makes sense for t — > 0, is known |Du 99j . The left hand side is 
difficult. 

In 1978, R. Dudley proved Conjecture Q for classes of {0, l}-valued functions, 
where it clearly reads as follows: F is a uniform Donsker class if and only if its 
combinatorial dimension v{F,l) is finite (see |Du 99j . |LTj 14.3). As Talagrand 
mentiones in |LTj 14.3, this remarkable characterization is one of the main results 
on the combinatorics of empirical processes on {0, 1} classes. 

In his 1992 Inventiones paper, Talagrand proved Conjecture^for convex classes, 
however up to an additional factor of log^\l/t) in the integrand |T 92j . In 2002, 
Talagrand rewrote his proof in |T 02j . now valid for arbitrary classes, and asked 
about the optimal value of the absolute constant exponent M. 
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In their recent Inventiones paper |MV 03j , Mendelson and Vershynin reduced the 
exponent of the logarithm in Talagrand's inequahty to M = 1/2. This was achieved 
by a new argument, via construction of an unbalanced separated tree in F. However, 
these methods alone could not possibly remove the factor of y^log(l/t). 

The logarithmic factor is removed completely in the forthcoming preprint jRVj. 
Thus the optimal exponent is M = and Conjecture^is true. Proving this requires, 
in addition to the combinatorial ingredients of |MV 03| . a new iteration argument 
and a general probabilistic reduction scheme. Along the proof of Conjecture Q its 
quantitative form would be especially helpful in practical applications. It can be 
written in words as follows: 

In Dudley's entropy integral inequality, the entropy can be replaced by 
the combinatorial dimension. 

The argument is based on the comparison of the combinatorial dimension and the 
uniform entropy D{F,t) (also known as Koltchinskii-Pollard entropy), which is the 
supremum of the metric entropies of F with respect to the metric of -Z>2(m) over all 
probabilities fi. Even though the combinatorial dimension and the uniform entropy 
are not equivalent in general, the situation changes dramatically when we look at 
regular classes, those for which, say, v {F,2t) < lv{F,t) for ah t > 0. Talagrand 
proved the required equivalence for these classes, however with an additional factor 
of log*^(l/t), where M is an absolute constant (T fl2j . In the Inventiones paper 
|MV 03 j of Mendelson and Vershynin, the exponent was reduced to M = 1 even 
without the regularity asumption on the class. In general, this exponent is optimal. 

In |RVj we prove that with the regularity assumption, the logarithmic factor can 
be removed completely; thus the optimal exponent in Talagrand's result is M = 0, 
hence 

the uniform entropy and the combinatorial dimension are equivalent for 
all regular classes. 

The argument is similar to that for Conjecture^ the iteration scheme works in both 
cases, for the integral and for the regular quantity. 
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